Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

We investigate whether Deep Reinforcement Learning (Deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies in dynamic environments. We used Deep RL to train a humanoid robot with 20 actuated joints to play a simplified one-versus-one (1v1) soccer game. We first trained individual skills in isolation and then composed those skills end-to-end in a self-play setting. The resulting policy exhibits robust and dynamic movement skills such as rapid fall recovery, walking, turning, kicking and more; and transitions between them in a smooth, stable, and efficient manner - well beyond what is intuitively expected from the robot. The agents also developed a basic strategic understanding of the game, and learned, for instance, to anticipate ball movements and to block opponent shots. The full range of behaviors emerged from a small set of simple rewards. Our agents were trained in simulation and transferred to real robots zero-shot. We found that a combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training in simulation enabled good-quality transfer, despite significant unmodeled effects and variations across robot instances. Although the robots are inherently fragile, minor hardware modifications together with basic regularization of the behavior during training led the robots to learn safe and effective movements while still performing in a dynamic and agile way. Indeed, even though the agents were optimized for scoring, in experiments they walked 156% faster, took 63% less time to get up, and kicked 24% faster than a scripted baseline, while efficiently combining the skills to achieve the longer term objectives. Examples of the emergent behaviors and full 1v1 matches are available on the supplementary website.Comment: Project website: https://sites.google.com/view/op3-socce

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks

The United States COVID-19 Forecast Hub dataset

Academic researchers, government agencies, industry groups, and individuals have produced forecasts at an unprecedented scale during the COVID-19 pandemic. To leverage these forecasts, the United States Centers for Disease Control and Prevention (CDC) partnered with an academic research lab at the University of Massachusetts Amherst to create the US COVID-19 Forecast Hub. Launched in April 2020, the Forecast Hub is a dataset with point and probabilistic forecasts of incident cases, incident hospitalizations, incident deaths, and cumulative deaths due to COVID-19 at county, state, and national, levels in the United States. Included forecasts represent a variety of modeling approaches, data sources, and assumptions regarding the spread of COVID-19. The goal of this dataset is to establish a standardized and comparable set of short-term forecasts from modeling teams. These data can be used to develop ensemble models, communicate forecasts to the public, create visualizations, compare models, and inform policies regarding COVID-19 mitigation. These open-source data are available via download from GitHub, through an online API, and through R packages

The James Webb Space Telescope Mission

Twenty-six years ago a small committee report, building on earlier studies, expounded a compelling and poetic vision for the future of astronomy, calling for an infrared-optimized space telescope with an aperture of at least $4m$. With the support of their governments in the US, Europe, and Canada, 20,000 people realized that vision as the $6.5m$ James Webb Space Telescope. A generation of astronomers will celebrate their accomplishments for the life of the mission, potentially as long as 20 years, and beyond. This report and the scientific discoveries that follow are extended thank-you notes to the 20,000 team members. The telescope is working perfectly, with much better image quality than expected. In this and accompanying papers, we give a brief history, describe the observatory, outline its objectives and current observing program, and discuss the inventions and people who made it possible. We cite detailed reports on the design and the measured performance on orbit.Comment: Accepted by PASP for the special issue on The James Webb Space Telescope Overview, 29 pages, 4 figure

A Comparison Study of Static Mapping Heuristics for a Class of Meta-tasks on Heterogeneous Computing Systems

Heterogeneous computing (HC) environments are well suited tomeet the computational demands of large, diverse groups of tasks (i.e., a meta-task). The problem of mapping (de ned as matching and scheduling) these tasks onto the machines of an HC environment has been shown, in general, to be NP-complete, requiring the development of heuristic techniques. Selecting the best heuristic to use in a given environment, however, remains a di cult problem, because comparisons are often clouded by di erent underlying assumptions in the original studies of each heuristic. Therefore, a collection of eleven heuristics from the literature has been selected, implemented, and analyzed under one set of common assumptions. The eleven heuristics examine

Identifying and Predicting the Goals and Concerns Prioritized by Individuals with Inflammatory Bowel Disease

Background and Aims: In order to provide high-quality care, providers need to understand their patients’ goals and concerns. This study aims to identify and predict the goals and concerns prioritised by patients with inflammatory bowel disease [IBD] in the outpatient setting. Methods: Mixed-methods analysis was performed to identify the types, frequencies, and predictors of IBD patients’ goals and concerns using 4873 surveys collected over 2016–2019 at 25 gastroenterology clinics across the USA participating in the Crohn’s & Colitis Foundation’s IBD Qorus Learning Health System. Results: Patients with IBD most often prioritised goals and concerns related to symptoms/disease activity [50%] and clinical course/management [20%], whereas psychosocial/quality of life [12%] and medication [6%] concerns were less frequent. Females (odds ratio [OR] 22.1, 95% confidence interval [CI] 5.3–91.5) and patients in clinical remission [OR 2.2, 95% CI 1.2–4.1] were more likely to prioritise family planning. Patients >60 years old [OR 3.1, 95% CI 1.5–6.5] and patients with active disease [OR 3.2, 95% CI 1.4–7.6] were more often concerned about travelling. Smokers were more often concerned about nutrition [OR 4.2, 95% CI 1.9–9.2]. Surgery was more often a concern of patients with perianal Crohn’s disease [OR 2.1, 95% CI 1.2–3.5], active disease [OR 1.9, 95% CI 1.1–3.4], and those with recent hospitalisations [OR 2.5, 95% CI 1.2–5.4]. Conclusions: IBD patients prioritised the remission of physical symptoms as treatment goals and they were less frequently concerned about medications and their side effects. Patients’ demographics, IBD characteristics, and health care utilisation patterns can predict specific types of concerns/goals

The Reliability of Patient Self-reported Utilization in an Inflammatory Bowel Diseases Learning Health System

Background: Inflammatory bowel disease (IBD) care is beset with substantial practice variation. Learning health systems (LHSs) aim to learn from this variation and improve quality of care by sharing feedback and improvement strategies within the LHS. Obtaining accurate information on outcomes and quality of care is a priority for LHS, which often includes patients' self-reported data. While prior work has shown that patients can accurately report their diagnosis and surgical history, little is known about their ability to self-report recent healthcare utilization, medication use, and vaccination status. Methods: We compared patient self-reported data within the IBD Qorus LHS regarding recent IBD-related emergency department (ED) visits, hospitalizations, computerized tomography (CT) scans, corticosteroid use, opioid use, influenza vaccinations, and pneumococcal vaccinations with electronic health record (EHR) data. Results: We compared 328 patient self-reports to data extracted from the EHR. Sensitivity was moderate-to-high for ED visits, hospitalizations, and CT scans (76%, 87%, and 87%, respectively), sensitivity was lower for medication use with 71% sensitivity for corticosteroid use and only 50% sensitivity for self-reported use of opioids. Vaccinations were reported with high sensitivity, but overall agreement was low as many patients reported vaccinations that were not registered in the EHR. Conclusions: Self-reported IBD-related ED visits, hosp